Multi-channel lidar sensor module

ABSTRACT

The present invention relates to a multi-channel lidar sensor module capable of measuring at least two target objects using one image sensor. The multi-channel lidar sensor module according to an embodiment of the present invention includes at least one pair of light emitting units configured to emit laser beams and a light receiving unit formed between the at least one pair of emitting units and configured to receive at least one pair of reflected laser beams which are emitted from the at least one pair of light emitting units and reflected by target objects.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation-in-part of International Application No.PCT/KR2017/012365 filed on Nov. 3, 2017, which claims priority toRepublic of Korea Patent Application No. 10-2017-0097950 filed on Aug.2, 2017, which are incorporated by reference herein in their entirety.This application also claims priority to Republic of Korea PatentApplication No. 10-2019-0017703 filed on Feb. 15, 2019, which isincorporated by reference in its entirety.

BACKGROUND 1. Technical Field

The present invention relates to a lidar sensor module, and moreparticularly, to a multi-channel lidar sensor module capable ofmeasuring at least two target objects using one image sensor.

2. Description of the Prior Art

Light amplification by the stimulated emission of radiation (LASER)causes stimulated emission of light to amplify the light and irradiate alaser.

Light detection and ranging (LiDAR) is technology for measuring adistance using a laser. The LiDAR has been developed into a form forbuilding topography data for building three-dimensional geographicinformation system (GIS) information and visualizing the builttopography data. Accordingly, the LiDAR has been applied in aconstruction field, a defense field, and the like.

FIG. 1 is a view illustrating a lidar sensor module according to therelated art.

As shown in FIG. 1, the lidar sensor module according to the related artincludes a light emitting unit 10 and a light receiving unit 20. Thelight emitting unit 10 includes a laser diode 11 configured to emit alaser beam and a collimating lens 12 configured to condense and convertthe emitted laser beam into a parallel beam. The light receiving unit 20includes a condensing lens 21 configured to condense and convert a laserbeam reflected by an obstacle (target object) A into a parallel beam andan image sensor 22 configured to receive the laser beam condensed by thecondensing lens 21. A band pass filter 23 is disposed between thecondensing lens 21 and the image sensor 22 to transmit only a reflectedlaser beam in a wavelength range emitted from the light emitting unit 10and to not transmit beams in other wavelength ranges.

In the case of the lidar sensor module of the related art as configuredabove, one light emitting unit and one light receiving unit constituteone lidar sensor module. Accordingly, the lidar sensor module of therelated art may sense one target object A existing on a light sourceoptical axis and measure a distance to the target object A.

On the other hand, the lidar sensor module is used to detect surroundingobstacles in autonomous vehicles, autonomous driving drones, and thelike. For example, when the lidar sensor module is used in theautonomous vehicle, a plurality of lidar modules are required to detectthe front, rear, left, and right of the vehicle. In addition, since itis necessary to detect upper and lower ends with respect to onedirection, at least two lidar sensor modules are also required for thesame direction.

As described above, since the lidar sensor module of the related art candetect/measure only one target object A existing on the light sourceoptical axis, when n channels are required, n lidar sensor modules arerequired, which increases costs and an occupied space.

SUMMARY

The present invention is directed to providing a multi-channel lidarsensor module capable of measuring two target objects using one imagesensor.

The present invention is directed to providing an object informationacquiring apparatus which acquires distance information and typeinformation related to an object using a single sensor.

Technical solutions of the present invention may not be limited to theabove, and other technical solutions of the present invention will beclearly understandable to those having ordinary skill in the art fromthe disclosures provided below together with the accompanying drawings.

According to an embodiment of the present invention, a multi-channellidar sensor module includes at least one pair of light emitting unitsconfigured to emit laser beams and a light receiving unit formed betweenthe at least one pair of emitting units and configured to receive atleast one pair of reflected laser beams which are emitted from the atleast one pair of light emitting units and reflected by target objects.

The at least one pair of light emitting units may be disposed in avertical direction or in parallel in a horizontal direction with respectto the ground.

The light receiving unit may include a condensing lens configured tocondense the at least one pair of reflected laser beams and an imagesensor configured to receive the condensed laser beams from thecondensing lens, and one laser beam of the at least one pair ofreflected laser beams is received in one region of the image sensor andthe other laser beam of the at least one pair of reflected laser beamsis received in the other region of the image sensor.

The at least one pair of light emitting units may be provided with aplurality of pairs of light emitting units, each of the pairs of lightemitting units may be disposed around the light receiving unit and mayface the light receiving unit, and the light emitting units providedwith the plurality of pairs of light emitting units may be controlledsuch that emission periods thereof do not overlap each other.

The multi-channel lidar sensor module may further include an opticalfilter unit configured to adjust transmittance of the reflected laserbeam received by the light receiving unit.

The optical filter unit may be an optical film having a preset size andshape, and a transmittance slope may be formed on a surface of theoptical film such that transmittance is adjusted according to a distancebetween the light emitting unit and the target object.

Transmittance of a central portion of the optical film may be highest,and transmittance may be gradually decreased in a direction from thecentral portion to a peripheral portion of the optical film.

The light receiving unit may include a condensing lens configured tocondense the reflected laser beam, an image sensor configured to receivethe condensed laser beam from the condensing lens, and a band passfilter formed between the condensing lens and the image sensor, whereinthe optical filter unit is formed by being applied on a surface of theband pass filter or a surface of the image sensor.

The optical filter unit may be formed by applying a coating materialsuch that transmittance is highest at a central portion of the surfaceof the band pass filter or the surface of the image sensor and applyingthe coating material such that transmittance is gradually decreased in adirection toward a peripheral portion of the surface of the band passfilter or the surface of the image sensor.

According to an embodiment of the present invention, an objectinformation acquiring apparatus may be provided. The apparatus mayacquire object information including type information and distanceinformation related to an object, include a camera module configured tocapture a periphery thereof; a laser module spaced apart from the cameramodule in a direction of a perpendicular axis and configured to emit alaser beam in a form of a line extending in a direction of a horizontalaxis; and a controller configured to acquire a first image captured bythe camera module at an emission timing of the laser module and a secondimage captured by the camera module at a non-emission timing of thelaser module, acquire, when the first image is captured, distanceinformation related to an object included in the first image based on aposition in the direction of the perpendicular axis, at which the laserbeam is received on the first image, and when the second image iscaptured, acquire type information related to an object included in thesecond image based on a pixel value of the second image.

According to another embodiment, a method of acquiring objectinformation may be provided. The method may be performed by an objectinformation acquiring apparatus including a camera module configured tocapture a periphery thereof and a laser module spaced apart from thecamera module in a direction of a perpendicular axis and configured toemit a laser beam in a form of a line extending in a direction of ahorizontal axis, include acquiring a plurality of images including afirst image captured by the camera module at an emission timing of thelaser module and a second image captured by the camera module at anon-emission timing of the laser module; acquiring distance informationrelated to an object included in the first image based on a position inthe direction of the perpendicular axis, at which the laser beam isreceived on the first image; based on the acquired distance information,determining whether a distance from the object information acquiringapparatus to the object is within a predetermined distance; and when thedistance from the object information acquiring apparatus to the objectis within the predetermined distance as a determination result,acquiring type information related to an object included in the secondimage based on a pixel value of the second image.

According to still another embodiment, a method of acquiring objectinformation may be provided. The method may be performed by an objectinformation acquiring apparatus including a camera module configured tocapture a periphery thereof and a laser module spaced apart from thecamera module in a direction of a perpendicular axis and configured toemit a laser beam in a form of a line extending in a direction of ahorizontal axis, include acquiring a plurality of images including afirst image captured by the camera module at an emission timing of thelaser module and a second image captured by the camera module at anon-emission timing of the laser module; acquiring type informationrelated to an object included in the second image based on a pixel valueof the second image; determining whether an object having apredetermined classification value is included in the second image; andwhen the object having the predetermined classification value isincluded in the second image, acquiring distance information related toan object included in the first image based on a position in thedirection of the perpendicular axis, at which the laser beam is receivedon the first image.

According to still another embodiment, a multi-channel lidar sensormodule may be provided. The module may comprise a light emitting unitincluding at least one pair of emitting units for emitting laser beams;and a light receiving unit formed between the at least one pair ofemitting units and configured to receive at least one pair of reflectedlaser beams that are emitted from the at least one pair of emittingunits and reflected by a target object.

Herein, the at least one pair of light emitting units may be disposed ina vertical direction or in parallel in a horizontal direction withrespect to the ground.

Herein, the light receiving unit may include a condensing lensconfigured to condense the at least one pair of reflected laser beamsand an image sensor configured to receive the condensed laser beams fromthe condensing lens, and one laser beam of the at least one pair ofreflected laser beams may be received in one region of the image sensorand the other laser beam of the at least one pair of reflected laserbeams is received in the other region of the image sensor.

Herein, the at least one pair of light emitting units may be providedwith a plurality of pairs of light emitting units, each of the pairs oflight emitting units is disposed around the light receiving unit andfaces the light receiving unit, and the light emitting units providedwith the plurality of pairs of light emitting units may be controlledsuch that emission periods thereof do not overlap each other.

Herein, the multi-channel lidar sensor module may further comprise anoptical filter unit configured to adjust transmittance of the reflectedlaser beam received by the light receiving unit.

Herein, the optical filter unit may be an optical film having a presetsize and shape, and a transmittance slope may be formed on a surface ofthe optical film such that transmittance is adjusted according to adistance between the light emitting unit and the target object.

Herein, transmittance of a central portion of the optical film may behighest, and transmittance may be gradually decreased in a directionfrom the central portion to a peripheral portion of the optical film.

Herein, the light receiving unit may include: a condensing lensconfigured to condense the reflected laser beam; an image sensorconfigured to receive the condensed laser beam from the condensing lens;and a band pass filter formed between the condensing lens and the imagesensor, wherein the optical filter unit may be formed by being appliedon a surface of the band pass filter or a surface of the image sensor.

Herein, the optical filter unit may be formed by applying a coatingmaterial such that transmittance is highest at a central portion of thesurface of the band pass filter or the surface of the image sensor andapplying the coating material such that transmittance is graduallydecreased in a direction toward a peripheral portion of the surface ofthe band pass filter or the surface of the image sensor.

Technical solutions of the present invention may not be limited to theabove, and other technical solutions of the present invention will beclearly understandable to those having ordinary skill in the art fromthe disclosures provided below together with the accompanying drawings.

According to a multi-channel lidar sensor module according to anembodiment of the present invention, it is possible to provide amulti-channel lidar sensor module including one light receiving unit anda plurality of light emitting units. Therefore, it is possible to detecta plurality of target objects (A) existing on a plurality of lightsource optical axes and measure distances to the target objects usingone multi-channel lidar sensor module.

In addition, the plurality of target objects (A) can bedetected/measured using one multi-channel lidar sensor module, therebyconsiderably reducing costs of purchasing a plurality of lidar sensormodules and solving a space problem caused by the plurality of lidarsensor modules.

Furthermore, transmittance of a reflected laser beam received by thelight receiving unit can be adjusted according to a distance to thetarget object, thereby performing accurate measurement on multipleregions.

That is, a light amount of a laser beam received by an image sensor canbe uniformly maintained at a certain level due to a difference of thelight amount according to a distance to the target object beingminimized, thereby performing accurate measurement on both a neardistance region and a long distance region.

According to an embodiment, distance information and type informationrelated to an object can be acquired using a single sensor.

Effects of the present invention may not be limited to the above, andother effects of the present invention will be clearly understandable tothose having ordinary skill in the art from the disclosures providedbelow together with accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view illustrating a lidar sensor module according to therelated art.

FIGS. 2 and 3 are views illustrating a multi-channel lidar sensor moduleaccording to an embodiment of the present invention.

FIG. 4 is a view illustrating an application example of themulti-channel lidar sensor module according to the embodiment of thepresent invention.

FIG. 5 is a graph showing an emission period in which a laser beam isemitted from a plurality of light emitting units.

FIG. 6 is a view illustrating an operation state when an emission periodis T1.

FIG. 7 is a view illustrating an operation state when an emission periodis T2.

FIGS. 8 to 11 are views illustrating various application examples of themulti-channel lidar sensor module according to the embodiment of thepresent invention.

FIG. 12 is a view illustrating a multi-channel lidar sensor moduleaccording to another embodiment of the present invention.

FIGS. 13 and 14 are views illustrating an optical filter unit of themulti-channel lidar sensor module according to another embodiment of thepresent invention.

FIG. 15 is a block diagram illustrating an object information acquiringapparatus according to an embodiment of the present invention.

FIG. 16 is a block diagram illustrating a controller according to theembodiment.

FIG. 17 is a diagram illustrating a method of acquiring distanceinformation of the object information acquiring apparatus according tothe embodiment.

FIG. 18 is a stereoscopic diagram illustrating the object informationacquiring apparatus according to the embodiment.

FIG. 19 is a side view illustrating the object information acquiringapparatus according to the embodiment.

FIGS. 20 to 24 are diagrams illustrating an object information acquiringoperation performed by the object information acquiring apparatusaccording to various embodiments.

FIG. 25 is an image showing a display on which object information isdisplayed according to an embodiment.

FIG. 26 is an image showing a display on which object information isdisplayed according to another embodiment.

FIG. 27 is a diagram illustrating a control of emission timings of alaser module (100) and a light-emitting diode (LED) module (2000).

FIG. 28 is a diagram illustrating a sensing unit (3100) according to theembodiment.

FIG. 29 is a flowchart illustrating a method of acquiring objectinformation according to an embodiment.

FIG. 30 is a flowchart illustrating a method of acquiring objectinformation according to another embodiment.

DETAILED DESCRIPTION

The present invention has various modifications and embodiments, and thedescriptions of the present invention will be described along withspecific embodiments with reference to the accompanying drawings.However, it is not intended that the present invention is limited to thespecific embodiments, and it is to be interpreted that all theconversions, equivalents and substitutions belonging to the concept andtechnical scope of the present invention are included in the presentinvention.

Terms used in the present invention are used for the sake of describingthe specific embodiments and are not intended to limit the presentinvention. As used herein, the singular forms “a,” “an,” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be understood that terms such as“comprise,” “include,” and “have,” when used herein, specify thepresence of stated features, integers, steps, operations, elements,components, or combinations thereof, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, or combinations thereof. Hereinafter, amulti-channel lidar sensor module according to embodiments of thepresent invention will be described with reference to the drawings.

According to an embodiment, there may be provided an object informationacquiring apparatus, which acquires object information including typeinformation and distance information related to an object, includes: acamera module configured to capture a periphery thereof; a laser modulespaced apart from the camera module in a direction of a perpendicularaxis and configured to emit a laser beam in a form of a line extendingin a direction of a horizontal axis; and a controller configured toacquire a first image captured by the camera module at an emissiontiming of the laser module and a second image captured by the cameramodule at a non-emission timing of the laser module, and, when the firstimage is captured, acquire distance information related to an objectincluded in the first image based on a position in the direction of theperpendicular axis, at which the laser beam is received on the firstimage, and when the second image is captured, acquire type informationrelated to an object included in the second image based on a pixel valueof the second image.

Here, the camera module may include a sensing unit including a pluralityof sensing elements arranged in an array form the direction of theperpendicular axis.

The sensing unit may be divided into a first region and a second regiondifferent from the first region and may include a first sensor, which isprovided in the first region and acquires a laser beam image, and asecond sensor which is provided in the second region and acquires areflection image.

The controller may increase a threshold value of the sensing unit,acquire a third image captured by the sensing unit of which thethreshold value is increased at the emission timing of the laser module,and acquire distance information related to an object included in thethird image based on the third image.

The controller may include a distance calculation unit configured toacquire the distance information based on a pixel position of a laserbeam image on the first image and an object recognition unit configuredto acquire the type information based on the pixel value of the secondimage, wherein the laser beam image indicates a laser beam that isemitted from the laser module, reflected from the object, and thenreceived by the camera module.

The object recognition unit may include an artificial neural network.

The object information acquiring apparatus may further include alight-emitting diode (LED) module configured to emit light to the objectat the non-emission timing of the laser module such that accuracy of thetype information is improved.

When the object information acquiring apparatus is installed on a movingbody, the controller may generate a traveling control signal of themoving body based on the object information.

The moving body may be at least one of an automated guided vehicle, amobile robot, a vehicle, and an unmanned aerial vehicle.

According to another embodiment, there may be provided a method ofacquiring object information which is performed by an object informationacquiring apparatus including a camera module configured to capture aperiphery thereof and a laser module spaced apart from the camera modulein a direction of a perpendicular axis and configured to emit a laserbeam in a form of a line extending in a direction of a horizontal axis,includes: acquiring a plurality of images including a first imagecaptured by the camera module at an emission timing of the laser moduleand a second image captured by the camera module at a non-emissiontiming of the laser module; acquiring distance information related to anobject included in the first image based on a position in the directionof the perpendicular axis, at which the laser beam is received on thefirst image; determining, based on the acquired distance information,whether a distance from the object information acquiring apparatus tothe object is within a predetermined distance; and when the distancefrom the object information acquiring apparatus to the object is withinthe predetermined distance as a determination result, acquiring typeinformation related to an object included in the second image based on apixel value of the second image.

When the object information acquiring apparatus is installed on a movingbody, the method of acquiring the object information may further includegenerating a traveling control signal of the moving body based on theacquired distance information and type information.

According to still another embodiment, there may be provided a method ofacquiring object information which is performed by an object informationacquiring apparatus including a camera module configured to capture aperiphery thereof and a laser module spaced apart from the camera modulein a direction of a perpendicular axis and configured to emit a laserbeam in a form of a line extending in a direction of a horizontal axis,includes acquiring a plurality of images including a first imagecaptured by the camera module at an emission timing of the laser moduleand a second image captured by the camera module at a non-emissiontiming of the laser module; acquiring type information related to anobject included in the second image based on a pixel value of the secondimage; determining whether an object having a predeterminedclassification value is included in the second image; and when theobject having the predetermined classification value is included in thesecond image, acquiring distance information related to an objectincluded in the first image based on a position in the direction of theperpendicular axis at which the laser beam is received on the firstimage.

When the object information acquiring apparatus is installed in a movingbody, the method of acquiring the object information may further includegenerating a traveling control signal of the moving body based on theacquired distance information and type information.

There may be provided a recording medium which stores a program forperforming any one of the above-described methods of acquiring theobject information.

In the multi-channel lidar sensor module according to the embodiment ofthe present invention, at least two light emitting units may be coupledto one light receiving unit, and thus, the one light receiving unit maydetect a target object existing on a light source optical axis of the atleast two light emitting units and may measure a distance to the targetobject.

FIGS. 2 and 3 are views illustrating the multi-channel lidar sensormodule according to the embodiment of the present invention. As shown inFIGS. 2 and 3, the multi-channel lidar sensor module according to theembodiment of the present invention includes a first light emitting unit110, a second light emitting unit 120, and a light receiving unit 200(210 and 220)

The first light emitting unit 110 and the second light emitting unit 120emit laser beams on light source optical axes. The first light emittingunit 110 emits a laser beam along an optical axis L1, and the secondlight emitting unit 120 emits a laser beam along an optical axis L2.

The first light emitting unit 110 and the second light emitting unit 120include laser diodes 111 and 121 configured to generate a laser beam andcollimating lenses 112 and 122.

The laser diodes 111 and 121 are optical elements configured to generatea laser beam. The collimating lenses 112 and 122 are optical lensesconfigured to convert the laser beam generated by the laser diodes intoa parallel beam and disposed behind the laser diodes.

The light receiving unit 200 is disposed between the first lightemitting unit 110 and the second light emitting unit 120. The firstlight emitting unit 110 and the second light emitting unit 120 may bedisposed in a vertical direction or disposed in parallel in a horizontaldirection with respect to the ground. When the first light emitting unit110 and the second light emitting unit 120 are disposed in the verticaldirection, an upper region and a lower region may be sensed and measuredwith respect to the same direction, and when the first light emittingunit 110 and the second light emitting unit 120 are disposed in thehorizontal direction, a left region and a right region may be sensed andmeasured with respect to the same height.

The light receiving unit 200 includes a condensing lens 210 configuredto condense a laser beam reflected by a target object A and an imagesensor 220 configured to receive the condensed laser beam from thecondensing lens 210. In addition, the light receiving unit 200 includesa band pass filter (not shown) which is disposed between the condensinglens 210 and the image sensor 220 to transmit only a reflected laserbeam in a wavelength range emitted from the light emitting units 110 and120 and to not transmit beams in other wavelength ranges.

The light receiving unit 200 is formed between the first light emittingunit 110 and the second light emitting unit 120 and receives a firstreflected laser beam R1 and a second reflected laser beam R2 which arerespectively emitted from the first light emitting unit 110 and thesecond light emitting unit 120 and reflected by the target objects A onthe optical axes.

In this case, as shown in FIGS. 2 and 3, the first reflected laser beamR1, which is emitted from the first light emitting unit 110 and isreflected and the second reflected light beam R2, which is emitted fromthe second light emitting unit 120 and is reflected, pass through thecondensing lens 210 and are received by the image sensor 220. Due to ageometrical structure in which the first light emitting unit 110 and thesecond light emitting unit 120 are disposed in the vertical direction,the first reflected laser beam is received in an upper region of theimage sensor 220, and the second reflected laser beam is received in alower region of the image sensor 220. That is, a region of the imagesensor 220, in which two reflected laser beams are received, is dividedinto the upper region and the lower region, and thus, a two-channellidar sensor module capable of measuring two measurement regions may beimplemented using one light receiving unit 200. Since a light receivingregion of the image sensor 220 is divided, two reflected laser beams maybe received without interfering with each other.

When the first light emitting unit 110 and the second light emittingunit 120 are disposed in the horizontal direction, due to a geometricalstructure, the first reflected laser beam is received in a left region(or a right region) of the image sensor 220, and the second reflectedlaser beam is received in the right region (or the left region) of theimage sensor 220.

FIG. 4 is a view illustrating an application example of themulti-channel lidar sensor module according to the embodiment of thepresent invention. FIG. 5 is a graph showing an emission period in whicha laser beam is emitted from a plurality of light emitting units. FIG. 6is a view illustrating an operation state when an emission period is T1.FIG. 7 is a view illustrating an operation state when an emission periodis T2. FIGS. 8 to 11 are views illustrating various application examplesof the multi-channel lidar sensor module according to the embodiment ofthe present invention.

Hereinafter, the multi-channel lidar sensor module according to theembodiment of the present invention is not limited to two channels, andmay be expanded to a plurality of channels.

For example, as shown in FIG. 4, one light receiving unit 200 may bedisposed at a center, and first to fourth light emitting units 110 to140 may be disposed at equal intervals on two imaginary lines orthogonalto each other with respect to a center point of the light receiving unit200, thereby implementing a four-channel lidar sensor module.

In this case, since the first light emitting unit 110 and the secondlight emitting unit 120 have different light receiving regions in whichlight beams are received from an image sensor 220 and the third lightemitting unit 130 and the fourth light emitting unit 140 have differentlight receiving regions in which light beams are received from the imagesensor 220, the light beams may be received without interfering witheach other. However, since the light receiving region of the first lightemitting unit 110 overlaps the light receiving regions of the third andfourth light emitting units 130 and 140 and the light receiving regionof the second light emitting unit 120 overlaps the light receivingregions of the third and fourth light emitting units 130 and 140, thelight beams may interfere with each other. Thus, it may be difficult toperform accurate measurement.

Therefore, as shown in FIG. 5, a laser beam emission period T1 of thefirst light emitting unit 110 and the second light emitting unit 120 anda laser beam emission period T2 of the third light emitting unit 130 andthe fourth light emitting unit 140 may be controlled so as to notoverlap each other.

When an emission period is T1, only the first light emitting unit 110and the second light emitting unit 120 may be operated, and the thirdlight emitting unit 130 and the fourth light emitting unit 140 may notbe operated. Thus, as shown in FIG. 6, the first light emitting unit 110and the second light emitting unit 120 may measure upper and lowerregions without being interfered with the third light emitting unit 130and the fourth light emitting unit 140.

Similarly, when an emission period is T2, the first light emitting unit110 and the second light emitting unit 120 may not be operated, and thethird light emitting unit 130 and the fourth light emitting unit 140 maybe operated. Thus, as shown in FIG. 7, the third light emitting unit 130and the fourth light emitting unit 140 may measure left and rightregions without being interfered with the first light emitting unit 110and the second light emitting unit 120.

In addition, as shown in FIG. 8, first to fourth light emitting units110 to 140 are disposed at equal intervals in a vertical direction or ahorizontal direction with respect to one light receiving unit 200,thereby implementing a four-channel lidar sensor module.

In this case, since the first light emitting unit 110 and the secondlight emitting unit 120 have different light receiving regions in whichlight beams are received from an image sensor 220 and the third lightemitting unit 130 and the fourth light emitting unit 140 have differentlight receiving regions in which light beams are received from the imagesensor 220, the light beams may be received without interfering witheach other. However, since the light receiving region of the first lightemitting unit 110 overlaps the light receiving region of the third lightemitting unit 130 and the light receiving region of the second lightemitting unit 120 overlaps the light receiving region of the fourthlight emitting unit 140, the light beams may interfere with each other.Thus, it may be difficult to perform accurate measurement.

Even in this case, as shown in FIG. 5, the laser beam emission period T1of the first light emitting unit 110 and the second light emitting unit120 and the laser beam emission period T2 of the third light emittingunit 130 and the fourth light emitting unit 140 may be controlled so asto not overlap each other.

As described above, a plurality of pairs of light emitting unitsopposite to each other may be disposed around one light receiving unit200, and emission periods of the pairs of light emitting units may becontrolled to implement a multi-channel lidar sensor module. FIGS. 9 to10 illustrate various application examples of the multi-channel lidarsensor module according to the embodiment of the present invention.

On the other hand, as shown in FIG. 11, the multi-channel lidar sensormodule according to the embodiment of the present invention may beimplemented as a multi-channel lidar sensor module which includes nlight receiving units 200 a to 200 c and n+1 light emitting units 110 to140. In this case, the light receiving unit and the light emitting unitmay be alternately disposed in a vertical direction or a horizontaldirection, and one light receiving unit may receive reflected laserbeams which are emitted from two adjacent light emitting units and arereflected by target objects.

Here, when the n light receiving units 200 a to 200 c are concurrentlyoperated, a reflected laser beam received by any one light receivingunit may be interfered with a reflected laser beam received by anadjacent light receiving unit.

Therefore, in order to prevent the reflected laser beams received byadjacent light receiving units from interfering with each other, asshown in FIG. 11, when the light receiving units 200 a to 200 c and thelight emitting unit 110 to 140 are alternately disposed, the adjacentlight receiving units may be controlled such that only any one lightemitting unit thereof is operated.

For example, when the light receiving unit 200 a is operated, the lightreceiving unit 200 b may be controlled so as to not be operated, and thelight receiving unit 200 c may be controlled to be operated. Similarly,when the light receiving unit 200 b is operated, the light receivingunit 200 a and the light receiving unit 200 c may be controlled so as tonot be operated.

According to the multi-channel lidar sensor module according to theembodiment of the present invention as described above, it is possibleto provide a multi-channel lidar sensor module including one lightreceiving unit and a plurality of light emitting units.

Therefore, it is possible to sense a plurality of target objects A andmeasure distances to the target objects A existing on a plurality oflight source optical axes using one multi-channel lidar sensor module.The plurality of target objects A may be sensed/measured using onemulti-channel lidar sensor module, thereby considerably reducing costsof purchasing a plurality of lidar sensor modules and solving a spaceproblem caused by the plurality of lidar sensor modules.

Next, a multi-channel lidar sensor module according to anotherembodiment of the present invention will be described with reference toFIGS. 12 to 14. FIG. 12 is a view illustrating the multi-channel lidarsensor module according to another embodiment of the present invention,and FIGS. 13 and 14 are views illustrating an optical filter unit of themulti-channel lidar sensor module according to another embodiment of thepresent invention.

As shown in FIG. 12, the multi-channel lidar sensor module according toanother embodiment of the present invention includes a first lightemitting unit 110, a second light emitting unit 120, a light receivingunit 200, and an optical filter unit 300. Since the first light emittingunit 110, the second light emitting unit 120, and the light receivingunit 200 are the same as those in the above-described embodiment,detailed descriptions thereof will be omitted.

The optical filter unit 300 controls transmittance of a reflected laserbeam received by the light receiving unit 200 according to a distance toa target object A. The optical filter unit 300 may be composed of anoptical film having a preset size and shape or may be formed to beapplied on a surface of a band pass filter (not shown) or an imagesensor 220.

When the optical filter unit 300 is composed of the optical film, atransmittance slope is formed on a surface of the optical film such thattransmittance is adjusted according to a distance between the lightemitting units 110 and 120 and the target object. That is, thetransmittance slope is formed such that the transmittance is highest ata central portion B of the optical film and is gradually decreased in adirection from the central portion to a peripheral portion C of theoptical film.

Even when the surface of the band pass filter or the image sensor 220 isformed to be coated with the optical filter unit 300, a coating materialis applied such that a concentration of the coating material isnon-uniform to form a transmittance slope as described above. That is,the coating material is applied such that the transmittance is highestat a central portion of the surface of the band pass filter or the imagesensor 220, and the coating material is applied such that thetransmittance is gradually decreased in a direction toward a peripheralportion of the surface.

In the optical filter unit 300, as shown in FIG. 13, a transmittanceslope may be formed in a concentric shape, or as shown in FIG. 14, atransmittance slope may be formed in a linear shape. When laser beamsemitted from the light emitting units 110 and 120 are line beams, theoptical filter unit 300 may have the concentric shape as shown in FIG.13. When the laser beams emitted from the light emitting units 110 and120 are point beams, the optical filter unit 300 may have the linearshape shown in FIG. 14.

An operation of the multi-channel lidar sensor module according toanother embodiment of the present invention as configured above will bedescribed.

When a laser beam is transmitted (emitted) from the light emitting unit110 or 120, the emitted laser beam is reflected by a target object (oran obstacle), the reflected laser beam is condensed by a condensing lens210, and the condensed laser beam is received by the image sensor 220.The band pass filter may transmit only a reflected laser beam in awavelength range emitted from the light emitting unit 110 or 120.

In this case, a transmittance slope is formed in the optical filter unit300 such that transmittance is highest at a central portion thereof andis gradually decreased in a direction from the central portion to aperipheral portion thereof. Due to a geometrical structure, a laser beamreflected by a target object at a long distance passes through thecentral portion of the optical filter unit 300, and a laser beamreflected by a target object at a near distance passes through theperipheral portion.

Since the transmittance of the central portion is high, the laser beamreflected at the long distance has a small light amount but passesthrough the central portion so as to correspond to the hightransmittance. Since the transmittance of the peripheral portion is low,the laser beam reflected at the near distance passes through theperipheral portion so as to correspond to the low transmittance.

Accordingly, a light amount of a laser beam received by the image sensor220 may be uniformly maintained at a certain level due to a differenceof the light amount according to a distance to a target object beingminimized, thereby performing accurate measurement on both a neardistance region and a long distance region.

FIG. 15 is a block diagram illustrating an object information acquiringapparatus according to an embodiment of the present invention.

Referring to FIG. 15, an object information acquiring apparatus 10000may include a laser module 1000, a light-emitting diode (LED) module2000, a camera module 3000, and a controller 4000.

Hereinafter, each component of the object information acquiringapparatus 10000 will be described in detail.

The laser module 1000 may emit a laser beam toward an object.

The laser module 1000 may emit laser beams in various forms. Forexample, the laser module 1000 may emit a laser beam in a form of a lineextending along one axis. Alternatively, the laser module 1000 may emita laser beam in a form of a planar shape.

The laser module 1000 may include a light source unit 1100 and ascanning unit 1200.

The light source unit 1100 may generate a laser beam emitted from thelaser module 1000.

The light source unit 1100 may be provided with various kinds of lightsources. For example, the light source unit 1100 may be provided with alaser diode and a vertical cavity surface emitting laser (VCSEL).

The light source unit 1100 may be provided in various forms. Forexample, the light source unit 1100 may be provided with a plurality oflaser diodes arranged in an array form.

The light source unit 1100 may generate laser beams having variouswavelengths. For example, the light source unit 1100 may generate laserbeams having wavelengths of 850 nm, 905 nm, and 1,550 nm.

The scanning unit 1200 may generate an emission beam using a laser beamgenerated from the light source unit 1100. For example, the scanningunit 1200 may generate a laser beam in a form of a line from a laserbeam in a form of a point generated from the light source unit 1100.

The scanning unit 1200 may include various optical components. Forexample, the scanning unit 1200 may include a collimator configured togenerate parallel light. Alternatively, the scanning unit 1200 mayinclude a lens configured to diffuse a laser beam generated from thelight source unit 1100 in a uniaxial or biaxial direction. In addition,the scanning unit 1200 may include a mirror configured to reflect alaser beam generated from the light source unit 1100 to form an emissiondirection of the laser beam. In addition, the scanning unit 1200 mayinclude various optical components such as a metasurface includingnanopillars.

The LED module 2000 may emit light toward the object.

The LED module 2000 may emit light beams having various wavelengths. Forexample, the LED module 2000 may emit light beams having wavelengths of850 nm, 905 nm, and 1,550 nm.

The LED module 2000 may be provided in various forms. For example, theLED module 2000 may be provided with a plurality of LED elementsarranged in an array form. Alternatively, the LED module 2000 may beprovided with a plurality of LED elements arranged in an irregular form.

The camera module 3000 may photograph a periphery of the objectinformation acquiring apparatus 10000. Accordingly, the camera module3000 may acquire an image by photographing the periphery of the objectinformation acquiring apparatus 10000.

The camera module 3000 may include a sensing unit 3100 configured tosense light and a light receiving lens 3200 configured to guide light tothe sensing unit 3100.

The sensing unit 3100 may include various types of sensing elements. Forexample, the sensing unit 3100 may include a charge coupled device (CCD)and a complimentary metal oxide semiconductor (CMOS). Alternatively, thesensing unit 3100 may include a single-photon avalanche diode (SPAD) anda photodiode.

The sensing unit 3100 may be made of various materials. For example, thesensing unit 3100 may be made of silicon, germanium, or InGaAs.Accordingly, a wavelength band in which light receiving sensitivity ofthe sensing unit 3100 is maximized may be varied.

The sensing unit 3100 may generate an electrical signal using an opticalsignal.

The sensing unit 3100 may receive a laser beam that is emitted from thelaser module 1000 and then reflected from an object. Accordingly, thesensing unit 3100 may acquire an image including the received laserbeam.

The camera module 3000 may include a filter. The filter may be providedin various types. For example, the camera module 3000 may include aninfrared filter and a visible light filter.

The camera module 3000 may include an optical shutter including anoptical switch.

The controller 4000 may control each of the laser module 1000, the LEDmodule 2000, and the camera module 3000.

The controller 4000 may control an emission timing, intensity, and apulse rate of a laser beam emitted from the laser module 1000.

The controller 4000 may control an emission timing, intensity, and apulse rate of light emitted from the LED module 2000.

The controller 4000 may control light receiving sensitivity of thecamera module 3000. For example, the controller 4000 may control a gainvalue or a threshold value of the sensing unit 3100.

The controller 4000 may acquire an image captured by the camera module3000 from the camera module 3000.

The controller 4000 may acquire type information related to an objectincluded in the acquired image. For example, the controller 4000 mayacquire the type information related to the object based on a pixelvalue of the acquired image.

The controller 4000 may acquire the type information related to theobject through various methods. For example, the controller 4000 mayacquire the type information based on a look-up table. Here, the look-uptable may be provided based on the pixel value or intensity of theacquired image.

Alternatively, the controller 4000 may acquire the type informationrelated to the object using an artificial neural network (NN) that islearned to perform an image recognizing operation.

Meanwhile, the controller 4000 may perform a preprocessing operation onthe acquired image. For example, the preprocessing operation may includeedge detection, blurring, sharpening, and red, green, and blue (RGB)normalization.

The controller 4000 may acquire the type information related to theobject based on an image acquired through the preprocessing operation.

Meanwhile, although not shown, the object information acquiringapparatus 10000 may include a communication module. The controller 4000may acquire the type information related to the object from an externalserver through the communication module. In this case, an operation foracquiring the type information related to the object may be performed inthe server.

The controller 4000 may acquire distance information related to theobject based on a laser beam that is emitted from the laser module 1000,reflected by the object, and then is received by the camera module 3000.

For example, the controller 4000 may acquire the distance informationrelated to the object based on a position of the received laser beam onthe image acquired from the camera module 3000. In this case, thecontroller 4000 may acquire the distance information related to theobject in consideration of an installation position and a posture of thecamera module 3000, an angle of the sensing unit 3100 from a ground, andthe like.

Alternatively, the controller 4000 may acquire the distance informationrelated to the object based on a reception time of the laser beam.Specifically, the controller 4000 may acquire the distance informationrelated to the object based on a difference between an emission time atwhich a laser beam is emitted from the laser module 1000 and a receptiontime at which a laser beam is received by the camera module 3000.

In addition, the controller 4000 may acquire the distance informationrelated to the object based on a phase difference between the laserbeams. Specifically, the controller 4000 may acquire the distanceinformation related to the object based on a phase difference betweenthe laser beam emitted from the laser module 1000 and the laser beamreceived by the camera module 3000.

The controller 4000 may be provided as a micro controller unit (MCU) ora central processing unit (CPU). However, the present invention is notlimited thereto, and the controller 4000 may be provided as variouschips configured to perform an operation of acquiring the distanceinformation, which is performed in the present invention.

FIG. 16 is a block diagram illustrating the controller according to theembodiment.

The controller 4000 may include a distance calculation unit 4100 and anobject recognition unit 4200.

The distance calculation unit 4100 may calculate distance informationrelated to an object through various methods. For example, the distancecalculation unit 4100 may calculate the distance information related tothe object based on a triangulation method. Alternatively, the distancecalculation unit 4100 may calculate the distance information related tothe object based on a time-of-flight (TOF) method, a phase shift (PS)method, and a frequency modulated continuous wave (FMCW) method.

The controller 4000 may acquire an image captured by the camera module3000. For example, the controller 4000 may acquire a first image(including a laser) captured by the camera module 3000 at an emissiontiming of the laser module 1000 and a second image (not including alaser) captured by the camera module 3000 at a non-emission timing ofthe laser module 1000.

When the acquired image is the first image, the distance calculationunit 4100 may calculate distance information related to an objectincluded in the first image based on a position at which a laser beamemitted from the laser module 1000 is received on the first image.

Specifically, the distance calculation unit 4100 may detect a pixelposition of the received laser beam and may calculate a distance to theobject based on the detected pixel position. The distance calculationunit 4100 may detect the pixel position of the laser beam based on asize of a pixel value of the first image. Alternatively, when a lightintensity of a specific region is greater than a light intensity ofother regions in the first image, the distance calculation unit 4100 maydetect position information related to a pixel in a correspondingregion.

On the other hand, the first image may be an image captured at thenon-emission timing of the LED module 2000. Thus, accuracy of thedistance information may be improved. This is because, when the pixelposition of the laser beam is detected, light emitted from the LEDmodule 2000 may become noise. That is, an image by light excluding thelaser beam emitted from the laser module 1000 may become noise.

When the acquired image is the second image, the object recognition unit4200 may acquire type information related to an object included in thesecond image based on a pixel value of the second image. For example,the object recognition unit 4200 may acquire the type information basedon intensity of the second image. Alternatively, the object recognitionunit 4200 may acquire the type information based on an RGB value of thesecond image.

On the other hand, the second image may be an image captured at theemission timing of the LED module 2000. Thus, accuracy of the typeinformation may be improved. This is because the pixel value of thesecond image may be increased by the LED module 2000.

For example, at night, a size of an optical signal acquired by thesensing unit 320 may not be sufficient to recognize an object. In thiscase, an object recognition rate of the object recognition unit 4200 maybe decreased. That is, accuracy of type information related to an objectmay be decreased.

Here, as the LED module 2000 emits light toward the object, the pixelvalue of the second image may be increased. Accordingly, the objectrecognition rate of the object recognition unit 4200 may be increased.That is, the accuracy of the type information related to the object maybe increased.

Meanwhile, the object recognition unit 4200 may also acquire typeinformation related to the object included in the first image based onthe first image. However, in this case, accuracy of the type informationmay be decreased as compared with a case in which the type informationis acquired based on the second image. This is because, when an objectis detected, a laser beam emitted from the laser module 1000 may becomenoise.

The object recognition unit 4200 may acquire type information related toan object through various methods. For example, the object recognitionunit 4200 may recognize the object using a learned artificial NN.

The artificial NN may include various NNs. For example, the artificialNN may include a convolution NN (CNN) that extracts features using afilter. Alternatively, the artificial NN may include a recurrent NN(RNN) that has a structure in which output of a node is fed back asinput again. In addition, the artificial NN may include various types ofNNs such as a restricted Boltzmann machine (RBM), a deep belief network(DBN), a generative adversarial network (GAN), and a relation network(RN).

The artificial NN may be learned through various methods. For example,the artificial NN may include supervised learning, unsupervisedlearning, reinforcement learning, and imitation learning. In addition,the artificial NN may be learned through various learning methods.

FIG. 17 is a diagram illustrating a method of acquiring distanceinformation of the object information acquiring apparatus according tothe embodiment.

The laser module 1000 may emit laser beams to a first object Ob1 and asecond object Ob2 farther away from the object information acquiringapparatus 10000 than the first object Ob1.

The camera module 3000 may acquire a first laser beam emitted from thelaser module 1000 and then reflected from the first object Ob1 and asecond laser beam emitted from the laser module 1000 and then reflectedfrom the second object Ob2.

The first laser beam and the second laser beam may pass through thelight receiving lens 3200 and then be received by the sensing unit 3100.

The controller 4000 may acquire coordinates of a first point P1 that isa position at which the first laser beam is received on the sensing unit3100.

The controller 4000 may acquire distance information related to thefirst object Ob1 based on the coordinates of the first point P1. Forexample, the distance information may mean a distance from the lasermodule 1000 to the first object Ob1.

In this case, the controller 4000 may calculate the distance informationrelated to the first object Ob1 in consideration of a distance betweenthe laser module 1000 and the camera module 3000 in a direction of aperpendicular axis, an inclination angle of the sensing unit 3100 fromthe perpendicular axis, a distance between the sensing unit 3100 and thelight receiving lens 3200, and a size of the sensing unit 3100.

As in the first object Ob1, the controller 4000 may calculate distanceinformation related to the second object Ob2 based on coordinates of asecond point P2 which is a position at which the second laser beam isreceived on the sensing unit 3100.

As shown in FIG. 17, the first point P1 may be farther away from thelaser module 1000 than the second point P2. That is, as a position of alaser beam received on the sensing unit 3100 is moved farther away fromthe laser module 1000, a distance from an object reflecting the laserbeam to the object information acquiring apparatus may be decreased.

FIG. 18 is a stereoscopic diagram illustrating the object informationacquiring apparatus according to the embodiment.

Referring to FIG. 18, the object information acquiring apparatus 10000may include the laser module 1000, the LED module 2000, and the cameramodule 3000.

Meanwhile, the laser module 1000, the LED module 2000, and the cameramodule 3000 may be the same as or correspond to those described withreference to FIGS. 15 to 17. Therefore, detailed descriptions thereofwill be omitted.

The object information acquiring apparatus 10000 may include a substrate5000 having a flat plate shape and a base 6000 disposed parallel to thesubstrate 5000 and provided in a flat plate shape.

The substrate 5000 may be formed such that a length thereof along alongitudinal axis corresponding to a perpendicular axis V is greaterthan a length thereof along a lateral axis corresponding to a horizontalaxis H

The substrate 5000 may be bonded to the base 6000 so as to be parallelto the base 6000.

The base 6000 may have at least one connection groove.

When the object information acquiring apparatus 10000 is installed on amoving body, the object information acquiring apparatus 10000 may beinstalled on the moving body through the connection groove.

The laser module 1000 may be installed on the substrate 5000. Forexample, the laser module 1000 may be disposed to be spaced apart fromthe camera module 3000 in the direction of the perpendicular axis V.

The laser module 1000 may emit laser beams in various forms. Forexample, the laser module 1000 may emit a laser beam in a form of a lineextending in the direction of the horizontal axis H.

The camera module 3000 may be installed on the substrate 5000. Forexample, the camera module 3000 may be disposed to be spaced apart fromthe laser module 1000 in the direction of the perpendicular axis V.

The LED module 2000 may be installed on the substrate 5000.

The LED module 2000 may be provided at various positions. For example,the LED module 2000 may be disposed between the laser module 1000 andthe camera module 3000. In this case, the laser module 1000, the LEDmodule 2000, and the camera module 3000 may be disposed on an imaginarystraight line in the direction of the perpendicular axis.

Alternatively, the LED module 2000 may be disposed in a form whichsurrounds the camera module 3000.

Meanwhile, the LED module 2000 may be disposed closer to the cameramodule 3000 than the laser module 1000.

FIG. 19 is a side diagram illustrating the object information acquiringapparatus according to the embodiment.

Referring to FIG. 19, an object information acquiring apparatus 10000may include the laser module 1000, the LED module 2000, the cameramodule 3000, the substrate 5000, and the base 6000. In addition,although not shown, the object information acquiring apparatus 10000 mayinclude the controller 4000.

Meanwhile, components of the object information acquiring apparatus10000 may be the same as or correspond to those described with referenceto FIGS. 15 to 18. Therefore, detailed descriptions thereof will beomitted.

The laser module 1000 may be disposed at an upper end of the substrate5000.

The laser module 1000 may emit a laser beam in a form of a line.Specifically, the laser module 1000 may emit a laser beam in a form of aline extending along a reference axis perpendicular to the perpendicularaxis V and the horizontal axis H.

Therefore, when viewed from the perpendicular axis V, the emitted laserbeam may be formed in the form of the line perpendicular to thehorizontal axis H. Similarly, when viewed from the horizontal axis H,the emitted laser beam may be formed in the form of the lineperpendicular to the perpendicular axis V.

The light source unit 1100 may be installed perpendicular to thesubstrate 5000. Accordingly, the light source unit 1100 may emit a laserbeam in a direction perpendicular to the substrate 5000.

In this case, the laser beam emitted from the light source unit 1100 maybe reflected by the scanning unit 1200 and then emitted in the directionperpendicular to the substrate 5000. Here, the scanning unit 1200 may beprovided as a mirror. The laser beam emitted from the light source unit1100 may pass through a lens or a prism and then be emitted in thedirection perpendicular to the substrate 5000.

On the other hand, the light source unit 1100 does not necessarily emita laser beam in the direction perpendicular to the substrate 5000. Forexample, the light source unit 1100 may emit a laser beam in a directionparallel to the substrate 5000, and the scanning unit 1200 may refractthe laser beam emitted in the parallel direction in the directionperpendicular to the substrate 5000. Accordingly, the laser beam emittedfrom the laser module 1000 may be emitted in the direction perpendicularto the substrate 5000 when viewed from a side thereof.

The camera module 3000 may be disposed at a lower end of the substrate5000.

The camera module 3000 may receive a laser beam reflected from theobject.

The camera module 3000 may include the sensing unit 3100 disposed in anarray form in the direction of the perpendicular axis V. For example,the sensing unit 3100 may be provided with an avalanche photodiode(APD).

The light receiving lens 3200 may be disposed parallel to the substrate5000. That is, the light receiving lens 3200 may be disposed such that acentral axis of the light receiving lens 3200 is perpendicular to thesubstrate 5000.

Alternatively, the light receiving lens 3200 may be disposed such thatthe central axis of the light receiving lens 3200 is inclined from thehorizontal axis H to the laser module 1000 when viewed from a sidethereof.

The LED module 2000 may be disposed between the laser module 1000 andthe camera module 3000.

The controller 4000 may acquire distance information related to theobject based on a position at which the laser beam emitted from thelaser module 1000 and then reflected from the object is received on thesensing unit 3100. Specifically, the controller 4000 may calculate thedistance information related to the object based on a reception positionof the received laser beam in the direction of the perpendicular axis V.

Meanwhile, the camera module 3000 is illustrated in FIG. 19 as directlyreceiving the laser beam reflected from the object, but the presentinvention is not limited thereto. For example, the camera module 3000may also receive a laser beam that is emitted from the laser module1000, reflected from the object, and then reflected from a mirrorincluded in the object information acquiring apparatus 10000.

FIGS. 20 to 24 are diagrams illustrating an object information acquiringoperation performed by the object information acquiring apparatusaccording to various embodiments.

Specifically, FIG. 20 shows a first image captured by the camera module3000 at an emission timing of the laser module 1000. The emission timingmay refer to a time point at which a laser beam is emitted from thelaser module 1000.

The first image may include laser beam images corresponding to laserbeams that are emitted from the laser module 1000, reflected from aplurality of targets ta1, ta2, and ta3, and then received by the cameramodule 3000.

The controller 4000 may acquire the first image.

The distance calculation unit 4100 may acquire distance informationrelated to the target ta1, ta2, or ta3 included in the first image. Thedistance calculation unit 4100 may acquire the distance informationbased on a position of the laser beam image along a perpendicular axison the first image.

For example, the distance calculation unit 4100 may detect a pixelposition of the laser beam image. The distance calculation unit 4100 maycalculate the distance information related to the target ta1, ta2, orta3 based on the detected pixel position.

Meanwhile, the object recognition unit 4200 may acquire type informationrelated to the target ta1, ta2, or ta3. For example, the objectrecognition unit 4200 may acquire the type information based on a pixelvalue of the first image. On the other hand, the contents described withreference to FIGS. 15 to 19 may be applied to a method of acquiring typeinformation of the object recognition unit 4200, and thus, detaileddescriptions thereof will be omitted.

The controller 4000 may classify the target ta1, ta2, or ta3 so as tohave a predetermined classification value based on the acquired typeinformation. For example, a first target ta1, i.e., a person, and athird target ta3, i.e., a chair, may be classified to haveclassification values corresponding to obstacles, and a second targetta2, i.e., a charging station, may be classified to have a separateclassification value different from the classification values. Here, thesecond target ta2 may refer to an apparatus or place for charging amoving body equipped with the object information acquiring apparatus10000.

The classification value corresponding to the obstacle may bereclassified according to characteristics thereof. For example, theclassification value corresponding to the obstacle may be classifiedinto various classification values according to a risk. Alternatively,the classification value corresponding to the obstacle may be classifiedaccording to whether the obstacle is a moving body. For example, theclassification value of the first target ta1, i.e., a moving body, maybe different from the classification value of the third target ta3,i.e., a fixed object.

On the other hand, since it is impossible to recognize an object using aconventional distance measurement sensor, there is a problem in that itis impossible for a moving body mounted with the distance measurementsensor to travel accurately. For example, when the distance measurementsensor is mounted on an automatic guided vehicle (AGV) or a robotcleaner, the second target ta2 of the first image is detected as anobstacle, and thus, the AGV or the robot cleaner may bypass the secondtarget ta2 entirely.

In addition, there is a problem in that it is impossible to accuratelymeasure a distance with respect to an object using a conventional imagesensor.

On the other hand, the object information acquiring apparatus 10000 mayacquire not only accurate distance information but also type informationrelated to the target ta1, ta2, or ta3. Accordingly, when the objectinformation acquiring apparatus 10000 is installed in the AGV or therobot cleaner, it is possible for the AGV or the robot cleaner to travelefficiently.

For example, the AGV equipped with the object information acquiringapparatus 10000 may not recognize the second target ta2 as an obstaclesuch as to bypass the second target ta2 entirely but may enter thesecond target ta2 based on the type information related to the secondtarget ta2. Thus, the AGV may be recharged in the second target ta2.

The controller 4000 may generate a travelling control signal forcontrolling traveling of the moving body equipped with the objectinformation acquiring apparatus 10000 based on acquired objectinformation. For example, when an obstacle around the moving body isdetected, the controller 4000 may generate a traveling signal to allowthe moving body to bypass the obstacle.

On the other hand, it may be impossible to efficiently generate atraveling signal using a conventional one-dimensional distancemeasurement sensor. For example, when the distance measurement sensorsenses the first target ta1, a moving body mounted with the distancemeasurement sensor may recognize two feet or legs of the first targetta1 as a separate obstacle. Accordingly, when a width of the moving bodymounted with the distance measurement sensor is less than a distancebetween the two feet of the first target ta1, a traveling signal toallow the moving body to pass between the two feet of the first targetta1 may be generated. The generated traveling signal is a travelingsignal generated without regard to the characteristics or type of thefirst target ta1. Thus, there may be a problem in that the moving bodycollides with the first target ta1 when the first target ta1 moves.

On the other hand, the object information acquiring apparatus 10000 mayacquire the type information related to the first target ta1, and thus,the controller 4000 may generate a traveling signal to allow the objectinformation acquiring apparatus 10000 to bypass the first target ta1without passing between the two feet of the first target ta1 based onthe type information.

In addition, the object information acquiring apparatus 10000 maygenerate a traveling signal of the moving body in consideration of aheight, width, and size of an object included in an image acquiredthrough the camera module 3000.

On the other hand, when the object recognition unit 4200 calculatesdistance information based on the laser beam image, the remaining imageof the first image excluding the laser beam image may become noise. Theremaining image may refer to a reflection image.

Thus, in order to improve accuracy of the distance information, thecontroller 4000 may acquire a second image in which intensity of thenoise is reduced from the first image.

FIG. 21 is a diagram shows the second image.

For example, the controller 4000 may generate the second image using thefirst image by adjusting a threshold value of the sensing unit 3100.

Alternatively, the controller 4000 may generate the second image usingthe first image by blurring the remaining image in the first image

As shown in FIG. 20, the distance calculation unit 4100 may acquiredistance information related to the target ta1, ta2, or ta3 included inthe second image.

Accuracy of second distance information acquired from the second imagemay be greater than accuracy of first distance information acquired fromthe first image.

Meanwhile, the controller 4000 may acquire a third image captured by thecamera module 3000 at a non-emission timing of the laser module 1000.The non-emission timing may refer to a time point at which a laser beamis not emitted from the laser module 1000.

FIG. 22 is a diagram showing the third image.

As shown in FIG. 22, the third image may not include the laser beamimage.

The controller 4000 may acquire the third image.

The object recognition unit 4200 may acquire type information related toeach of the target ta1, ta2, and ta3 included in the third image. Forexample, the object recognition unit 4200 may acquire the typeinformation based on a pixel value of the third image. Meanwhile, thecontents described with reference to FIG. 20 may be applied to a methodof acquiring type information of the object recognition unit 4200, andthus, detailed descriptions thereof will be omitted.

On the other hand, accuracy of third distance information acquired basedon the third image may be greater than the accuracy of the firstdistance information calculated based on the first image. This isbecause, when the object recognition unit 4200 acquires the typeinformation related to the target ta1, ta2, or ta3, the third image doesnot include the laser beam image that may become noise.

On the other hand, when a pixel value of an image captured from thecamera module 3000 is insufficient due to a surrounding environment (forexample, a weather situation), the accuracy of the type information maybe decreased.

The controller 4000 may operate the LED module 2000 to improve theaccuracy of the type information.

The controller 4000 may acquire a fourth image captured by the cameramodule 3000 at a non-emission timing of the laser module 1000 and anemission timing of the LED module 2000. The non-emission timing of thelaser module 1000 may refer to a time point at which a laser beam is notemitted from the laser module 1000. In addition, the emission timing ofthe LED module 2000 may refer to a time point at which light is notemitted from the LED module 2000.

FIG. 23 is a diagram showing the fourth image.

A pixel value of the fourth image may be larger than that of the thirdimage.

The fourth image may be an image with an increased pixel value.Alternatively, the fourth image may be an image with increasedbrightness.

The object recognition unit 4200 may acquire type information related toa target ta1, ta2, or ta3 included in the fourth image based on thefourth image.

Accordingly, accuracy of fourth type information acquired from thefourth image may be greater than the accuracy of the third typeinformation acquired from the third image.

On the other hand, the controller 4000 may acquire a fifth image inwhich only edges of the targets ta1, ta2, and ta3 remain from the thirdimage or the fourth image.

FIG. 24 is a diagram showing the fifth image.

The controller 4000 may detect edges of the targets included in thethird image based on the pixel value of the third image. The controller4000 may acquire the fifth image based on the detected edges.

Similarly, the controller 4000 may detect edges of the targets includedin the fourth image based on the pixel value of the fourth image. Thecontroller 4000 may acquire the fifth image based on the detected edges.

The object recognition unit 4200 may acquire type information related tothe target ta1, ta2, or ta3 based on the fifth image.

Accordingly, accuracy of the type information acquired from the objectrecognition unit 4200 may be improved.

On the other hand, in FIGS. 20 to 24, a wavelength band at which lightreceiving sensitivity of the sensing unit 3100 is maximized may be aninfrared band. That is, each of the first to fifth images may be animage acquired by an infrared sensor. In this case, the objectinformation acquiring apparatus 10000 may include an optical filterwhich transmits only light corresponding to a wavelength band of a laserbeam emitted from the laser module 1000 and blocks light having otherwavelength bands.

Alternatively, the wavelength band at which the light receivingsensitivity of the sensing unit 3100 is maximized may be a visible lightband. That is, the first to fifth images may be images acquired by avisible light sensor. In this case, the object information acquiringapparatus 10000 may include an optical filter which transmits only lightcorresponding to the visible light band and blocks light having otherwavelength bands.

The method of acquiring the object information performed by thecontroller 4000 has been described above.

The object information acquired by the controller 4000 may be displayedon an external display communicating with the object informationacquiring apparatus 10000 through various methods.

FIG. 25 is an image showing a display on which object information isdisplayed according to an embodiment.

The display may output an image including object information acquired bythe controller 4000. For example, the display may output an around viewwith respect to a moving body M as shown in FIG. 26. The objectinformation acquiring apparatus 10000 may be installed in the movingbody M. The display may be installed on the moving body M.

The around view image may be expressed differently according to adistance from the moving body M. For example, a distance of an objectmay be classified into a short distance, a medium distance, and a longdistance, which have a certain range. The around view image may bedisplayed in different colors according to the classified distances.

The display may display distance information and type informationrelated to the object so as to be displayed at a position of the objecton the around view image. For example, the distance information and thetype information may be displayed so as to be written in the forms ofnumerals and characters at the position of the object on the around viewimage.

The around view image may be generated by the controller 4000.Alternatively, the around view image may be generated by other devicescommunicating with the object information acquiring apparatus 10000.

A type of and distance from an object around the moving body M may bedisplayed on an image output through the display.

In another example, the display may output object information based onan image acquired by the camera module 3000.

FIG. 26 is an image showing a display on which object information isdisplayed according to another embodiment.

The display may output an image including object information acquired bythe controller 4000. For example, the display may output objectinformation based on an image acquired by the camera module 3000 at anon-emission timing of the laser module 1000.

The display may output an image, on which distance information and typeinformation related to an object around the object information acquiringapparatus 10000 are displayed, on the image acquired by the cameramodule 3000.

On the other hand, the controller 4000 may control emission timings ofthe laser module 1000 and the LED module 2000.

FIG. 27 is a diagram illustrating a control of the emission timings ofthe laser module 1000 and the LED module 2000.

The controller 4000 may control the emission timing of the laser module1000. For example, when the LED module 2000 does not emit light, thecontroller 4000 may control the laser module 1000 to emit a laser beam.That is, while the LED module 2000 emits light, the controller 4000 maycontrol the laser module 1000 to not emit a laser beam.

This is because, when the distance calculation unit 4100 calculatesdistance information based on a laser beam emitted from the laser module1000, light emitted from the LED module 2000 may become noise.

The controller 4000 may control the emission timing of the LED module2000. For example, when the laser module 1000 does not emit a laserbeam, the controller 4000 may control the LED module 2000 to emit light.This is because, when the object recognition unit 4200 performs a typeinformation acquiring operation, a laser beam emitted from the lasermodule 1000 may become noise.

A first period tt1 which is a duration time during which the lasermodule 1000 outputs a laser beam may be different from a second periodtt2 which is a duration time during which the LED module 2000 emitslight. For example, the first period tt1 may be less than the secondperiod tt2. This is because power consumption for emitting the laserbeam of the laser module 1000 may be higher than power for emitting thelight of the LED module 2000.

The number of instances of laser beam emission of the laser module 1000during one frame may be different from the number of instances of lightemission of the LED module 2000 during one frame. For example, thenumber of instances of the laser beam emission may be greater than thenumber of instances of the light emission. Alternatively, the number ofinstances of the light emission may be greater than the number ofinstances of the laser beam emission.

On the other hand, an emission timing of the laser module 1000 and anemission timing of the LED module 2000 may overlap each other. That is,at the same time point, a laser beam may be emitted from the lasermodule 1000, and light may be emitted from the LED module 2000. In thiscase, the controller 4000 may acquire distance information and typeinformation related to an object included in an image based on the imagecaptured at the same time by the camera module 3000. The controller 4000may increase a threshold value of the sensing unit 3100 to removeambient light as well as noise.

On the other hand, as shown in FIG. 28, the sensing unit 3100 may bedivided into a first region and a second region. For example, a firstsensor 3110 configured to acquire a reflection image may be provided inthe first region, and a second sensor 3120 configured to acquire a laserbeam image may be provided in the second region.

Light receiving sensitivity of the first sensor according to awavelength may be different from light receiving sensitivity of thesecond sensor according to a wavelength. For example, the lightreceiving sensitivity of the first sensor may be maximized in a visiblelight band, and the light receiving sensitivity of the second sensor maybe maximized in an infrared band.

The first sensor may include an infrared ray (IR) filter for blocking aninfrared ray. The second sensor may include a filter for blockingvisible light.

The controller 4000 may control a time during which the sensing unit3100 is exposed to light. For example, the controller 4000 may control ashutter of the camera module 3000 such that the sensing unit 3100 isexposed at an emission timing at which the laser module 1000 emits alaser beam.

The controller 4000 may control an exposure time of each of the firstsensor 3110 and the second sensor 3120. For example, the controller 4000may control the camera module 3000 such that the second sensor 3120 isexposed at the emission timing at which the laser module 1000 emits thelaser beam. Alternatively, the controller 4000 may control the cameramodule 3000 such that the first sensor 3110 is exposed at an emissiontiming at which the LED module 2000 emits light.

Meanwhile, the controller 4000 may perform a type information acquiringoperation based on distance information related to an object or mayperform a distance information acquiring operation based on typeinformation related to the object.

FIG. 29 is a flowchart illustrating a method of acquiring objectinformation according to an embodiment.

The method of acquiring the object information may include acquiringtype information related to an object included in an image captured by acamera module (S100), determining whether an object having apredetermined classification value is included in the image (S200), andwhen the object having the predetermined classification value isincluded in the image, acquiring distance information related to theobject using a laser module (S300).

First, an object recognizing unit 420 may acquire type informationrelated to the object included in the image captured by the cameramodule 3000 (S100). The acquiring of the type information may beperformed in the same manner as the method of acquiring the typeinformation described with reference to FIGS. 15 to 28, and thus,detailed descriptions thereof will be omitted.

A controller 4000 may determine whether the object having thepredetermined classification value is included in the image (S200). Forexample, the predetermined classification value may refer to aclassification value corresponding to an obstacle. Alternatively, thepredetermined classification value may refer to a classification valuecorresponding to a charging station. The predetermined classificationvalue may be set by a user.

When the object having the predetermined classification value isincluded in the image, a distance calculation unit 4100 may calculatethe distance information related to the object using the laser module1000 (S300). Specifically, the distance calculation unit 4100 maycalculate the distance information based on an image including a laserbeam emitted from the laser module 1000, wherein the image is acquiredby the camera module 3000. Meanwhile, a method of acquiring the distanceinformation may be performed in the same manner as the method ofacquiring the distance information described with reference to FIGS. 15to 28, and thus, detailed descriptions thereof will be omitted.

For example, only when an object corresponding to an obstacle isincluded in the image, the controller 4000 may operate the laser module1000 to calculate distance information related to the object.Accordingly, the controller 4000 may prevent energy consumption foroutputting an unnecessary laser beam in an obstacle-free environment. Inaddition, it is possible to reduce a calculation time of acquiringobject information of an object information acquiring apparatus 10000.

FIG. 30 is a flowchart illustrating a method of acquiring objectinformation according to another embodiment.

The method of acquiring the object information may include acquiringdistance information related to an object using a laser module (S110),determining, based on the distance information, whether a distance froma lidar device to the object is within a predetermined distance (S210),and when the distance to the object is within the predetermined distanceas a determination result, acquiring type information related to theobject (S310).

First, a controller 4000 may acquire the distance information related tothe object using the laser module 1000 (S110). Specifically, a distancecalculation unit 4100 may calculate distance information related to anobject included in an image based on the image acquired at an emissiontiming of the laser module 1000. Meanwhile, a method of acquiring thedistance information may be performed in the same manner as the methodof acquiring the distance information described with reference to FIGS.15 to 28, and thus, detailed descriptions thereof will be omitted.

Based on the distance information, the controller 4000 may determinewhether the distance from the lidar device to the object is within thepredetermined distance. For example, the predetermined distance may beset by a user. Alternatively, the predetermined distance may be changedaccording to an installation environment of an object informationacquiring apparatus 10000.

When the distance to the object is within the predetermined distance asthe determination result, an object recognition unit 4200 may acquirethe type information related to the object (S310).

For example, only when the object is positioned a short distance awayfrom the object information acquiring apparatus 10000, the objectrecognition unit 4200 may acquire the type information related to theobject. Here, the controller 4000 may operate an LED module 2000,thereby improving accuracy of the type information.

On the contrary, only when the object is not positioned at a shortdistance from the object information acquiring apparatus 10000, thecontroller 4000 may deactivate the LED module 2000. Alternatively, theobject recognition unit 4200 may not perform a type informationacquiring operation.

Accordingly, power consumption due to an operation of the LED module2000 may be reduced.

When the object having the predetermined classification value isincluded in the image, the distance calculation unit 4100 may calculatethe distance information related to the object using the laser module1000 (S300). Specifically, the distance calculation unit 4100 maycalculate the distance information based on an image including a laserbeam emitted from the laser module 1000, wherein the image is acquiredby the camera module 3000. Meanwhile, a method of acquiring the distanceinformation may be performed in the same manner as the method ofacquiring the distance information described with reference to FIGS. 15to 28, and thus, detailed descriptions thereof will be omitted.

For example, only when an object corresponding to an obstacle isincluded in the image, the controller 4000 may operate the laser module1000 to calculate distance information related to the object.Accordingly, the controller 4000 may prevent energy consumption tooutput an unnecessary laser beam in an obstacle-free environment.

The above-described object information acquiring apparatus 10000 mayinclude a light detection and ranging (LiDAR) device. Alternatively, theabove-described object information acquiring apparatus 10000 may beprovided as the lidar device.

Although, a case in which the object information acquiring apparatus10000 is installed in the moving body has been mainly described, thepresent invention is not limited thereto, and the object informationacquiring apparatus 10000 may be installed at any designated place.

For example, the object information acquiring apparatus 10000 may beinstalled at a certain place around a door to acquire object informationrelated to an object around the door. In this case, the objectinformation acquiring apparatus 10000 may be installed in a positionwhich has a certain angle.

The object information acquiring apparatus 10000 may be used forsecurity. For example, the object information acquiring apparatus 10000may acquire distance information and type information related to anobject in a designated surveillance region.

Although the embodiment of the present invention has been described, itwill be appreciated by those skilled in the art that the presentinvention may be variously modified and changed by adding, changing, orremoving constituent components without departing from the scope of thepresent invention described in the claims, and the modifications orchanges fall within the scope of the present invention.

What is claimed is:
 1. An object information acquiring apparatusconfigured to acquire object information including type information anddistance information related to an object, the apparatus comprising: acamera module configured to capture a periphery thereof; a laser moduleconfigured to emit a laser beam in a form of line shape extending in adirection of a horizontal axis, wherein the laser module is disposed tobe spaced apart from the camera module in a direction of a perpendicularaxis; and a controller configured to acquire a first image captured bythe camera module at an emission timing of the laser module and a secondimage captured by the camera module at a non-emission timing of thelaser module, acquire when the first image is acquired, distanceinformation related to an object included in the first image based on aposition of the laser beam in the first image along the perpendicularaxis, and when the second image is acquired, acquire type informationrelated to an object included in the second image based on a pixel valueof the second image.
 2. The object information acquiring apparatus ofclaim 1, wherein the camera module includes a sensing unit including aplurality of sensing elements arranged in an array form along theperpendicular axis.
 3. The object information acquiring apparatus ofclaim 2, wherein the sensing unit is divided into a first region and asecond region different from the first region, includes a first sensorwhich is provided in the first region and acquires a third imagereflecting the laser beam, and a second sensor which is provided in thesecond region and acquires a fourth image reflecting the object.
 4. Theobject information acquiring apparatus of claim 2, wherein thecontroller acquires a third image captured when the sensing unit has asecond threshold value larger than a first threshold value, wherein thefirst threshold value is a threshold value of the sensing unit at a timewhen the first image is captured, wherein the controller acquiresdistance information related to an object included in the third imagebased on the third image.
 5. The object information acquiring apparatusof claim 1, wherein the controller includes a distance calculation unitconfigured to acquire the distance information based on a pixel positionof a laser beam image on the first image and an object recognition unitconfigured to acquire the type information based on the pixel value ofthe second image, wherein the laser beam image indicates a laser beamthat is emitted from the laser module, reflected from the object, andthen received by the camera module.
 6. The object information acquiringapparatus of claim 5, wherein the object recognition unit includes anartificial neural network.
 7. The object information acquiring apparatusof claim 1, further comprising a light-emitting diode (LED) moduleconfigured to emit light to the object at the non-emission timing of thelaser module such that accuracy of the type information is improved. 8.The object information acquiring apparatus of claim 1, wherein, when theobject information acquiring apparatus is installed on a moving body,the controller generates a traveling control signal of the moving bodybased on the object information.
 9. The object information acquiringapparatus of claim 8, wherein the moving body is at least one of anautomated guided vehicle, a mobile robot, a vehicle, and an unmannedaerial vehicle.
 10. A method of acquiring object information to beperformed by an object information acquiring apparatus comprising acamera module configured to capture a periphery thereof and a lasermodule configured to emit a laser beam in a form of line shape extendingin a direction of a horizontal axis, wherein the laser module isdisposed to be spaced apart from the camera module in a direction of aperpendicular axis, the method comprising: acquiring a first imagecaptured by the camera module at an emission timing of the laser moduleand a second image captured by the camera module at a non-emissiontiming of the laser module; acquiring distance information related to anobject included in the first image based on a position of the laser beamalong the perpendicular axis on the first image; determining whether adistance from the object information acquiring apparatus to the objectis within a predetermined distance; and when the distance from theobject information acquiring apparatus to the object is within thepredetermined distance, acquiring type information related to an objectincluded in the second image based on a pixel value of the second image.11. The method of claim 10, further comprising, when the objectinformation acquiring apparatus is installed on a moving body,generating a traveling control signal of the moving body based on theacquired distance information and the type information.
 12. A method ofacquiring object information to be performed by an object informationacquiring apparatus comprising a camera module configured to capture aperiphery thereof and a laser module configured to emit a laser beam ina form of line shape extending in a direction of a horizontal axis,wherein the laser module is disposed to be spaced apart from the cameramodule in a direction of a perpendicular axis, the method comprising:acquiring a first image captured by the camera module at an emissiontiming of the laser module and a second image captured by the cameramodule at a non-emission timing of the laser module; acquiring typeinformation related to an object included in the second image based on apixel value of the second image; determining whether an objectcorresponding to a predetermined classification value is included in thesecond image; and when the object corresponding to a predeterminedclassification value is included in the second image, acquiring distanceinformation related to an object included in the first image based on aposition of the laser beam along the perpendicular axis on the firstimage.
 13. The method of claim 12, further comprising, when the objectinformation acquiring apparatus is installed on a moving body,generating a traveling control signal of the moving body based on theacquired distance information and the type information.